Model ing with Structures in Statistical Machine Translation

Most statistical machine translation systems employ a word-based alignment model. In this paper we demonstrate that word-based alignment is a major cause of translation errors. We propose a new alignment model based on shallow phrase structures, and the structures can be automatically acquired from parallel corpus. This new model achieved over 10% error reduction for our spoken language translation task. 1 I n t r o d u c t i o n Most (if not all) statistical machine translation systems employ a word-based alignment model (Brown et al., 1993; Vogel, Ney, and Tillman, 1996; Wang and Waibel, 1997), which treats words in a sentence as independent entities and ignores the structural relationship among them. While this independence assumption works well in speech recognition, it poses a major problem in our experiments with spoken language translation between a language pair with very different word orders. In this paper we propose a translation model that employs shallow phrase structures. It has the following advantages over word-based alignment: • Since the translation model can directly depict phrase reordering in translation, it is more accurate for translation between languages with different word (phrase) orders. • The decoder of the translation system can use the phrase information and extend hypothesis by phrases (multiple words), therefore it can speed up decoding. The paper is organized as follows. In section 2, the problems of word-based alignment models are discussed. To alienate these problems, a new alignment model based on shallow phrase structures is introduced in section 3. In section 4, a grammar inference algorithm is presented that can automatically acquire the phrase structures used in the new model. Translation performance is then evaluated in section 5, and conclusions are presented in section 6. 2 W o r d b a s e d A l i g n m e n t M o d e l In a word-based alignment translation model, the transformation from a sentence at the source end of a communication channel to a sentence at the target end can be described with the following random process: 1. Pick a length for the sentence at the target end. 2. For each word position in the target sentence, align it with a source word. 3. Produce a word at each target word position according to the source word with which the target word position has been aligned. IBM Alignment Model 2 is a typical example of word-based alignment. Assuming a sentence s = S l , . . . , s t at the source of a channel, the model picks a length m of the target sentence t according to the distribution P ( m I s) = e, where e is a small, fixed number. Then for each position i (0 < i _< m) in t, it finds its corresponding position ai in s according to an alignmen t distribution P(a i l i, a~ -1, m, s) = a(ai l i, re, l). Finally, it generates a word ti at the position i of t from the source word s~, at the aligned position ai, according to a translation z 1 m distribution P( t i ] t~, a 1 , s) -t(ti I s~,).